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Historically, there has been a gradual but distinct trend toward 

, , J! 

the vesting of more control in the federal and state government. Localities 
have not surrendered their autonon\y, but have become increasingly embedded 
In a multi-tier system, where stato and federal control* and funding of local 
organizations has risen substantially in the past twenty years (see Zucker, 
1980)/ While this trend is apparent in education, local school districts 
have been more successful than most public organizations dn retaining their 
autonomy (Kirst, 1970; Meyer, 1979). 

Schools are crucial to the pursuit of central societal goals, and 
therefore have been subjected to institutional definitions of what is proper 
educational procedure and practice. At the federal, state and local level, 
school performance is monitored, evaluated^ and overall assessments are 
made. Since the impltinentation of Title I of the Elementary and Secondary 
Education Act in 1965,' local school districts have been required to evaluate 
their performance. Further pressure.for evaluation is rooted in the in* 
creasingly heavy dependence, especially in some states, on other types of 
extra-local funding for school districts. These sources of funding increase 
the legitimacy of external^ demands for assessment; local district needs for 
evaluation to improve prograifi performance have largely been eclipsed by the 
need for accountability to funding sources. The current "rational view" of 
schools'^' held by these outsiders pos^s a problem for schools because of the 
legitimate authority the federal and state evaluation requirements have over 
Tnternll school practices. "Hyper-rational ization^* (Wise, 1977a and b) 




could be Ignored were 1t not for the concomitant attempt to enforce this 
view of educational process by each separate funding body. 

Conformity to institutional rules can be seen as the raison d'etre 
schools. Many of the processes 1n schools involve compliance to external 
• definitions of what schools should do, and responding to external mandates 
for evidence of such compliance. Especially as state and federal funds 
have increasingly been allocated to local public schools, these external 
demands for evidence have become more well defined and complex. Largely 
in response, local public school districts have established evaluation 
units. Such, units are not required and are generally not funded directly 
by federal or state sources but the requirements for evaluation data 
concerning Title T implementation Ccoupled with grant and special program , 
evaluatioiH requirements and heavy reporting requirements in soiuc siates) 
make it convenient for school districts to establi-ih an evaluation unit. 
Evaluation units generally serve to centralize and simplify data collec- 
tion and dissemination. ItJs_jio$ suprising that the nut:i>er of evaluation 
umtii in school districts ^has lit£:'a 11^ mushroomed since 1965. About a 
nh'rd of all school districts with over 5,000 students have evaluation 
units. Fully ?M of tne evaluation units have been established since 
1966 and over half since 1970 (Lyon, 1978; Lyon et al,, 1978). 

In this paper, a theory is developed which emphasizes the signal 
nature of evaluation units, Depending on the degree to which a school dis- 
trict exists in an institutionalized environment, the environment serves as 
a source of support Cincluding funds) for it. As a result, the environment 
tstats and federal government) exercises legitimate control and surveillance 
over the district's activities and performance (for a general disctssion. 



see Pfeffer & Salandk, 1978). From this perspective, the primary 
function of an evaluation unit Is to produce signals from the school 
dtstrtct to this external environment. Hence, most of the Information 
produced by the evaluation unit Is In response to accountability require- 
ments generated at the federal and state levels. Little of the Informa- 
tion Is used within the organization Cschool district)., though It may be 
disseminated within it Csee David. 1978, for further support .of this 
argument).. Further, the evaluation unit generally has Limited Internal 
' function Ce.g.. does not affect distnict policies or other units), with 
little connection between evaluation unit staff and instructional staff. 
In other words, the evaluation unit serves to legitimate what the school 
district. is doing by reporting it Cgeneral'ly in favorable terms) to 
external sources of funding and control to whom the district is 
legitimately accountable. 

A general theoretical explication of the sources and consequences 
bf institutional environments for organizations is developed in the next 
section. Then the concepts are applied more directly to evaluation units, 
in school districts. Some central hypotheses are tested using data drawn 
from a national . survey of evaluation units CLyon et al.. 1978). combined 
with measures of federal and state funding and control of school district 
organizations. • 

SOURCES AND CONSEQUENCES OF FORMAL STRUCTURE 
In schools, as in other large scale organizations, formal organiza- 
tional structures arise, often not clearly or closely related to the actual 
activities of the organization (cf. Gouldner. 1954). Thisformal structure 



Is fundamentally a blueprint for organizational activity. The organization 
chart, with its listing of major officers, departments, and programs, de- 
tails the organization's formally defined structure. These elements are 
linked by rationally defined connections between activities, impersonally 
embodied in the organization's explicit goals. 

. Much of mqdern organization theory has been concerned with exploring 
the sources of formal structure in organizations (Scott. 1975). Research 
thus far has investigated formal structure which arises primarily from ' 
problems of coordination inherent in the core technology (e.g.. Woodward. 
1965; Mohr. 1971; Comstock & Scott, 1977; Billings et al.. 1977). In 
contrast, the emerging theory of institutionalization CZucker. 1977a; 
' Meyer & Rowan. 1977) has stressed the role of social definition, of 
"myth." in determining organizational structure. Independent of the core 
technology, or "loosely coupled" to it (see Weick. 1976; Meyer & Rowan. 
1977).- formal structure created by social definition serves to legitimate 
tJ\e organization. The organization incorporates elements of structure 
such as evaluation units or affirmative action officers, which reaffirm " 
organizational conformity to the externally imposed definition .of what 
is legitimate. 

This body of wor'k on the sources of structure", whether task oriented 
•or Institutional, has treated the origin of formal structure as. largely 
outside of the organization's direct control: , the structure is determined 
either by the technology or by the institutionalized environment. ^It is 
important to note that wider societal definitions are involved in either " 



case. The application of appropriate technology is defined In terms of 
"stata-of-the-art," and organtzatlons are expected to adopt Innovations • 
deflned'as advances in technological development: Institutionalized 
procedures, certified professionals, and lirograms of action Involve 
actual organizational activity, but at the same time Incorporate 
societal definitions of proper practices even when conformity conflicts 
with: efficiency criteria (Meyer & Rowan, 1977). Regardless, then, of 
the particular advantage or disadvantage for the task performance itself, 
wide-spread definition of a technology, procedure, or division of departments 
as rational and legitimate leads to organizational adoption. For example, 
early adopters of civil service procedures exhibit characteristics which 
indicate their need for more formal personnel procedures, while cities 
adopting these procedures later in the process do not, adopting them ' 
simply on the basis of their widespread legitimacy (Tolbert & Zucker, 1980). 

But the organization itself should be recognized as an important deter- 
minant of its own location in the wider environment. Organizations are not 
simply passive captives of their environments, whether technological or 
^ institutional. Their role in regulatiiig environmental effects by developing 
boundary maintenance functions, domiin definitions, and other mechanisms of 
control has long been recognized in Organizatioa theory Csee Thompson,- 
1967). Most fundamentally, however, organizations seek to construct 
their own environments, not simply manage preexisting environmental con- 
straints Ce.g., Pfeffer's study of organizational merger, 1972). ' For 
example, organizations may define their appropriate institutional location 
COowling & Pfeffer, 1967): Tr^in • institutes wish to define themselves 
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as educational institutions, thereby gaining access to societal resources 
such as tax wite-offs and G, I. Bill flmds, although their functions, 
in fact, parallel personnel agencies more closely. 

The foregoing argument can be sunmarlzed as follov/s: 

Proposition 1: 

Organizations seek to define their own location in the wider 
institutional/technological environment. 

Organizations define their location along a continuum, ranging from (1) 
narrowly construing their activities as technical and refusing societal 
resources (e.g., training stipends for workers) which would broaden the 
societal relevance of their goals, or (2) broadly construing their 
activities as fundamentally societal and accepting societal resources 
which reinforce that view. 

This statement differs from earlier work on organizations in a 
number of respec1:s. Probably the most important, from an operational 
point of view, is; that the degree of dependence on societal resources . 
is a crucial step in defining the organization as one serving societal 
interests, and thcus- existing in an institutionalized environment. Hence, 
public organizations are, a priori , operating in institutionalized en- 
vironments, though the extent to which obligations of the organization 
are normatively defined may vary. - Organizations whifih are not public, 
but which* deliver services seen as relatedTtbs^e public good, may also 
operate! in irxStitutionalized environments. Increasingly, all organiza- 
tions, including profit -making firms Ce.g., Lockheed and Chrysler), are 
teing identified as central to the common good, and hence as deserving o 
of societal support, fl] 
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A corollary of central Importance, since organizations are bound by 
history and convention, Is: ' 

Corollary: Once an organizational type Is defined as societal 
or technical, other new organ1;[ation8 will have to demonstrate 
that they are not of that type before redefinition can succeed. 

Social definitions of appropriate procedures and practices have, 
over time, the force of facts. For example, edycationarorganizations 
are firmly embedded In the institutional environment; it is unlikely 
that an educational organization could successfully redefine its environ- 
ment as technological, escaping accreditation, certification, and other 
institutional requirements (Meyer & Rowan, 1978). 

i 

Consequences of Institutionally Derived Structure 

* Little work has focused on the consequences of formal organizational 
structure, whether derived from technological or institutional environ- 
ments. Generally, structure derived from needs of core technology is 
presumed to have consequences largely internal to the organization: 
altered task organization, altered efficiency (Leifer & Huber, 1977). 
But structure derived from institutional requirements is thought to be 
de-coupled from technical activities aad to affect, primarily, survival 
rates of organizations (Meyer & Roy^an, 1977)." ^ 

Generally, while organizations in the more "rational" approach to 
structure are seen as interpenetrated by the environment, most of these 
elements are ei their controlled or buffered to prevent them from having 

y ■ • , ■ 

any direct influence on Internal organizational functioning or structure 

# . ■ ^. 

CThompson, 1967). Typically, organizations are seen as autonomous, in- 
most senses not directly accountable to InterestsHocated outside 

■■ • , ' ' , • 

«* - ■ ' ^ 

. . ■ 11 



i 



the organization. While private and public organliatlons ^re cont1nu6usl.y 
Inter-penetrated by customers (e.g., purchasers of cars) or cllenti (e,g., 
student$ of schools), It has been noted generally that these groups do 
not exert much control over respective organizations (see Hazenfeld, 1972, 
on people processing organizations In general). While 1n principle such 
groups exert control because they must select the product or service, they 
frequently operate within limited choice ranges (one must have a car and, 
according to the amount which can he spent, may have a choice between two 
or three major competitors) or have a "captive audience" (requirements 
for school attendance coupled with local school concepts ensure, at bmt, 
a restricted range of choice). 

External Control 

The key to understanding the effects of the environment or internal 
organizational structure appears to be the locus, extent, and legitimacy . 
of externaKcontrol over internal organizational process. In normal mar- 
ket environments, organizations are viewed as legitimately autonomous in 
pursuit of their own goals as defined by them. In striking contrast to 
this, organizations in institutional environments a-e seen as properly 
accountable to societal interest^ and therefore as legitimately con- 
trolled by societal agents. To put it more directly, external adminis- 
trative and legislative control is thought to be desirable 'in order to 
ensure that such organizations are. indeed, serjong the societal Interests 
they were created to serve or perpetuate. 

Such external control necessitates the development of rules and pro- 
cedures, and also rather formal rules for evaluation. As Kaufman (1960) 



has pointed out, control without direct supervision (which has been 
termed "long range control") necessitates development of elaborate 
monitoring and evaluation systems and other control techniques, such as 
. professional i?--!*ion, to majcfmize "self-regulation." It should be noted 
that-all public organizations can legitimately be held accountable, since 
they are ostensibly organized for the public good. It is not the case, 
however, that all are actually so controlled. For example', when secrecy 
is essential (CIA, FBI) such control is relinquished. Further, some 
control is thought to be legitimate in private industry to the extent 
that the common good is potentially involved (e.g., railroads, air trans- 
portation, "vital resource" industries such as coal and,steel). - 

A second proposition can now bis stated: 

■ . '.. , ■ ■ - • > • ■ 

Proposition 2: 

The greater the extent to which an organization exists in 
aji insti-tUttonal environment, the, greater degree of legiti- 
mate control and demands for accountability coming from the 
relevant environment. 

The environmental control and demands for accountability are 
problematic to the extent that'direct surveillance of organizational 
perfonnance cannot be performed. As long as control is local, direct 
evidence can be bbtalned: But when funding and control come from extra- 
local sources, as infthe case of school districts, accountabil ity becofnes 
niore problematic. Effectively, the organization is required' to become 
"self-evaluating" CWIldavsky, 19721, and to transmit the information 

obtained to the ext'erna? sources/of funding and control. A tHird 
proposition, then, is: 
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^ Proposition 3: 

timate control and demands for accountability ar4 
extra-local, then organizations will be required to per- 
f cm evaluation (or, minimally, monitoring) of their 
own activities. 

At the same time, however, the support and funding the organization., 
requires may be contingent on the results of the evaluation. One major 
consequence of this is the creation of internal organizational uncertain- 
ties and inconsistencies. Loose coupling of administrative structure 
from the rest of the organization [and activities from each other) may, 
instead of being dysfunctional, be the most rational strategy for maxi- 
mizing production of positive evaluation results. Similarly, subunits 
responsible for responding to external demands for accountability will 

be loosely coupled to subunits performfng other activities. This poik 

I 

will' be- developed more fully below. 

Formal Structure and Signaling 

Much as individuals use signals in negotiating the labor marl^et Csee 
Spence, 1974), organizations use signals to.provide information to their 
envilronments. Organizations develop tasks, programs and goals, and even 
subunits to signal their intentions to the environment (on the latter point, 
see Meyer, 1980). Depending on' the control which the environment has over - 
the organization, the organization ^i 11 proliferate formal structure which 
produces signals demonstrating accountability to the relevant environment. 

In brief, * • * 

Proposition 4: 

To the extent that, the environment has legitimate control 



over an organization, the organijzation will produce struc- 
, ture which serves signaling function. t 

Even organizations which are largely autonomous from the institu- 
tional environment may, under some conditldns, be sufficiently constrained 
by it to produce structure which serves a signaling function. A case 
in point is the wideispread existence df anti -trust departments In cor- 
porations. However, organizations more deeply embedded in the institu- 
tional environment (tucker, 1977) have elaborated many aspects' of follnal 
structure which produce signals demonstrating accountability—evaluation 
units in schools, designed to transmit information concerning the quality 
and degree of task focus, and "management Information systems" (MIS units) 
in, \ocal employment and tif^ainlng programs, designed to transmit evidence 
of Internal competence in processing clients. 

Formal structure which serves signaling functions is, in many re- 
spects, equivalent to boundary personnel: It has the function of repre- 
senting the^ organization to the wider environment, of signaling its 
adequate performance as a means of generating additional resources, but^ 
has little internal responsibility (see Zucker, 1979). [2] Formal 
structure which serves largely boundary functions appears loosely coupled 
to the jest of the organizational structure precisely because it is not 
directly responsible for, or not focused on, -internal task performanlfe. 
Hence, ^ . 

Proposition 5: 

.• ^ < 
To the extent that component of formal structure are 
designed to send signals to the wider ins^titutional 
environment," they are loosejy coupled to internal organ- 
izational^performance. / 



■ Instltuttonal Structure and Evaluatton Criteria ; 

What, then, of the content of the formal structure and the activity , 
it generate?.? First, the content of formal" structure created as a response 
to non-institutional, technological environments .is primarily internal 
and taslc-focused. while that cheated as a response to instit*itional 

r environments is primarily externally focused. Hence. 

«♦ - 

Proposition 6: 

f?ZLf*''"-*"'^ generated in response to technological con- 
inS^ wh ?! concern«id witK internal .organizational function- 
I?3n%? rntr°"^^7^^1/*=*r^ generated in response to'institu- 
. ll^^A^^i^i^^'^' °" representation of 

Second, the actual altivities generated 'by the formal structure will 
be different: Organizations operating in institutional environments, such 
as schools, define their scope of activity tq^require being less and less_. 
certain about more and more Ceifiphasizing ambiguity and uncertainty). '] 
while organizations not so externally accountable define their scope of 
activity to require being more^and more certain about less aijd less 
^ (emphasizing control and certainty). Schools (and simtlar organizations) 
heed to emphasize the uncertainty/ambiguity because they are held externally 
accourjtabl.e to different (and changing) standards of ^hat is institutionally 
proper. Business organizations are^ot held accountable- externally to the 
same degree and the external standards which do apply are more unifom 
(e.g.. anti-trust regulations). ' " 

Further, the greater the interpenetration of the local organizations 
by institutional demands and resources, the more ambiguous and uncertain 
activities are presented: . - 
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propositi on 7: 

The greatisr the extent to which. /^n organization exists in an 
r, trvstitutlona] environment, and the external evaluation 
therefore becomes more critical, the organization stresses 
the amfciiguity of evaluation. criteria in order to make external 
evaluation more problematic. 

An important consequence of enhancing criteria^ is that the organization 
becomes less able to evaluate its own perfonnance. For example; befor? 
schools were heavily dependent on externa] (e;<tra-local ) support, vali- 
dation, and resources, they presented themselves as producing rather 
specific training, evaluated on grounds of. efficiency; as schools moved 
into a more instituttonalized environment, they defined their tasks as 
'increasingly ambiguous and uncertain, best evaluated on grounds of 
cfertiftcation (see Meyer & Rowan, 1975; compare Callahan, 1962 to 
Tyack, 1974). ' . . 

Organizations in institutional environments use the evaluation of 
task performance as a signal, alterable and ambiguous, to indicate coropli 
ahce to institutional rules, not to guide adjustments in task organizatio 
wganlzations in technological environments use evaluatiprwof task per- 
formance as an index of how well they are performing, as clear, unaltera- 
ble evidence which can be- used to guide adjustments in task organization. 
[3] Characteristics of the task itself determine the extent to which it 
can be- redefined as an index or a signal. Teaching, for example, is more 
ambiguous than autanobile assembly (for a general discussiian of active 
and intert tasks, see Dornbusch & Scbtt, 1975). Basically, it is. 
asserted that organizations in institutional environments will attempt 
to define their ^ask perfonnance in- a different way than organizations 



not held accountable to societal interests. Fundamen tally: 

(1) Organizations will act to produce and apply largely ambiguous 
criteria for task performance when societal support and funding depend 
on evaluation: 

(a) They will define the environment as more variable, increasing the 
scope of activities and making more aspects of the environment 
appear relevant to their tasks (e.g., range of pupil character i sties); 

(b) They will define tasks as more ambiguous and difficult (e.g., how can 
"good" teaching be defined, let alone regulated?); and ' 

(c) They win use evaluation information primarily as signals to the ex- ■ 
ternal environment. 

(2) Organizations act to reduce, or render inapplicable, institutional 

• . ; . ■ •' • . .\ 

rules when evaluation control is centered. in the local organizat°ional unit, 

3 - ■ ' . . J 

along. with resources (or reso^rcas are controlled by a lar^e and diffuse 
"public"): / . " 

(a) They will define the relevant environment to make it seem more consti^nt 
by reducing the scope of activities and range of .relevance (e.g., only 
serve one small segment of the market); 

(b) They will define tasks as clear, limiting the scope of the task (6.g., 
hot to produce a "good" product, but one which meets certain pre-set ' 

, s pec ilFi cations); and 
(c- They will use evaluation information primarily internally (it may,' in 
facii be secret or privileged), as indices of task performance. 
.Finally, the status of the organizational members of units responsible 
for evaluation will vary, depending on the degree to which evaluation 



serves a signaling function. Hence, 

Proposition 8: 

The greater the signaling function of evaluation, and 
hence, the more it is viewed as ambiguous, the lower 
the internal status of the members or unit responsible 
for evaluation. 

When evaluation simply serves to legitimate an organization, and is 
defined a<^ largely ambiguous and of little value to the organizational 
task performance, then the members or unit which produce it will be given 
minimal resources and support. ' - - 

In the next section, the relevance of this general model for evaluation 
units in school districts is, established. ' 

THE ROLE OF EVALUATION UfilTS IW SCHOOL DISTRICTS 

It is clear that educational organizations do not operate in ^he ra- 
tional technology-oriented fashion that most organizational theorists have 
assumed (see especially Meyer, 1977; Meyer & Rowan, 1977, 1978). Separate 
organ izatiiHial subcomponents and levels of authority do not articulate ° . 
wen with one\natjher but,' instead, operate relatively autonomously with 
-little control exercised (Weick, 1976 anji Cohen, March, & .01 sen, 1976, 
providi goofrdiscussions of these issues). The lack of control over cen- - 
tral task procesfSes has been labeled Ibose coupling. The primary reason 
for the presence of loose coupling in educational institutions' appears 
to^be that they function in environments which demand conformity to in- 
stitutional rules rather than to technological perfection or innovation, 
-with some subunits largely engaged in producing ^stgnals which indicate to 
the wider environment that the school is accountable to societal interests. 
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As Meyer (1977: 4-5) puts it, "...the technical organization faces in 
toward its technical core .and turns its back on the environment; the in- 
stitutional organization turns its^back on its technical core in order 
to concentrate on its institutional environment..; [the] crucial thing 
a school needs to do to survive is to conform to ^titutional rules - 
including coinnunity understandings - defining teacher categories and cre- 
dentials, pupil selection and definition, proper topics of instruction, 
and appropriate facilities." 

District-wide evaluation units have appeared, largely as a response 
to increased funding .and control at the -state and federal level, jto handle 
reporting, monitoring, and evatuation requirements. On a nationwide basis, 
85% of all evaluation units appeared following the implementation of federal 
evaluation requirements accompanying Title I in 1965 (David, 1978). In 
the Center for the Study of Evaluation survey, only ,3 out of 259 districts 
(1.2%) reported that they have no state or federally funded programs. 
State or federally funded programs are evaluated by the evaluation unit- in 
over 80% of the districts, while in less than 14% is this task delegated 
to other district units or personnel (and in less than 5% delegated to. out- 
side evaluators solely). - 

The direct relationship between the resources comingjnto the district 
and control is indicated in the following quote from .a director of Title I 
program (David, 1978: 13): "This district wi°ll accept all the strings that 
go with the Federal money." There is little doubt that" the primary function 
the evaluations serve is to meet state and federal reporting requirements, 
fs one evaluator expl icitly states, evaluation is generally associated witK 



accountability, not with information uSeful in assessing strengths 'and 
weaknesses of programs (David; 1978: 39): "I don't know whether tho.test 
scores are useful as a basis for making changes in the program because 
I don't deal with the content of the program." 

Before beginning a more extensive. analysis of the data on evaluation 
units collected in the Center for the Study of Evaluation survey, the cen- 
tral hypotheses, and their relationship to the propositions above, need to 
b| stated explicitly. First, as the percent of societal resources in- 
creases, the amount of regulation and control increases (Proposition 2). 
Sec6nd, as these resources are increasingly txtra-focal, the organization 
will proliferate, formal structure (units, positions) responsible for self- 
evaluation (Proposition 3). Third, as the degree ofi institutionalization 
and extra- local control ' increases, the percent of self-evaluation informa- 
tion used externally a1s« increases (the signal nature of evaluation, from / 
■proposit^ns 4 and 6). Fourth, as the information is increasingly used » 
externally, the unit, or personnel responsible for evaluation increasingly 
have minimal Influence over internal organisational policies, and have little 
connection to actua^ organizational tasks (loose coupl ing of uni^or per- 
sonnel from Proposition 5). Fifth, as ihe degree of extra-local control 
increases, the ambiguity of the information is stressed and a narrow 
range of evaluation takes place (Proposition 7). Finally, as the ambiguity 
and narrowness increases and as the. information use is more external, the 
status of the unit or personneV responsible fdr evaluation decreases (Prop- 
bsittdn 8) . 
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THE . TEST 

Sigce the analysis is preliminary at this point, full discussslon 
of, the- results awaits additional work with the full 4ata set. For now, 
a brief discussion of the sample, the variables created for this analysis, 
aid a very preliminary presentation of regression results serve to provide 
the framework for further work. 

The Sample 

• There" are two basic samples.;^ First, a postcard survey concerning 
presence or absence of evaluation units in a nationwide sample of school 
districts was conducted (Lyon, 1978). Second, a detailed survey instru- 
ment was sent to the heads of evaluation units in all districts having 
10,000 pupils or more (N=319h and to a 50% sample of districts having 
5,000 to 9,999 students "(N=573). Of these, 263 responded. While these 
districts varied widely on many of the characteristics measured by the 
survey instrument, there is little data on the characteristics of those 
districts which do not have evaluation units; Table 1 exhausts the 
Irifomation available. 

Variable Definition - ■ 

There are several sets of variables crucial to this analysis. First, 
organizational characteristics are used largely as given in the original 
data: (1) The staff and budget of the evaluation unit (EU Staff; EU Bud- 
get), and (2) the Budget *ahd assessed value of the entire district (Dis't. 
.Budget; D1st. Value). Second, the extent of local or external resources 
is defined in term§ of the percent white (highly negatively correlated. 



in turn, with free lunch provision and low student performance, both also 

good predictors of the inflow of federal and state funds)* Third, internal 
role of the evaluation unit. is assessed by a simple summing of the number 

of school policies the evaluation uni<t is said to effect (Table 2 
presents these marginals). Fourth, tight coupling is in terms of the dis- 
trict having some general guideline affecting instructional activities. 

o 

Fifth, the range of activities the evaluation unit engages in is defined 
as a simple sum (EU Act Range). 

The largest class of variables created are those which measure internal/ 

# ■ » 

external emphasis in terms of time the evaluation unit devotes and in terms 
of . the use of evaluation information. »^ince construction of these measures 
is complex, fuller discussion is deferred until the analysis is more com- 
plete (some of the measures will be reconstructed to better reflect the 
underlying variance). Basica.lly, each measure is constructed so as to 
either weight the relative use (Test Use: Ext is a ratio of external to 
internal use of both CR and NR tests) or sum across a number "of categories 
of Internal or external (e.g.. External Use). > 

Results ' ' ^. ■ ' ^ 

Evidence for the role of external funding or control in the creation 
of evaluation units can be found in the distribution of evaluation units 
across states (Lyon, 1978).' While in most states roughly a quarter to a 
third of the school districts have evaluation units, in states which have 
requirements for local evaluation or extensive state funding of schools, 
the percent having evaluation units is significantly higher (see Table 1). 
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While the s.ize of the district is important, with larger districts more 
likely to have both extensive external funding (especially in metropolitan^ 
districts) and evaluation units, the variance across states is at least as 
striking. While most states have no small (5,000. to 9,999 students) dis- 
tricts which have evaluation units, states with extensive funding or eval- 
uation requirements typically have evaluation units in a quarter "to a third 
of their small districts. 

Insert Table 1 here ♦ . . ' ' 

■ fj 

Therefore, while all schools exist in an institutional environment 
(Meyer & Rowan, 1975), the degree of institutionalization does vary as 
„a function of the differences in the amount of federal funds and regula- 
tions (depending on size, Ipcationr, and minority or poor enrollment) and 
as a function of state differences in funding and control over local 
public schools- As Propositions 4 and 6 above state, under these con- 
ditions, as institutionalization increases evaluation units are more 
likely to be established, since they constitute archetypical examples ^ 

o . , 

of formal structure which signals compliance to external demands far 

■ . "* 

accountability. 

To the extent that evaluation units serve as signals of compliance 
to external ^di recti ves and iflterests, their role in internal functioning 
of the organization is predicted to be minimal (Proposition 5). They 
are 'loosely coupled to the rest of the organizatipn, but not by accident 
(We1ck, 1976) - they are established to serve as a boundary unit of. the 
organization, to'negotiate a complex institutional environment, not to 
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guide the internal task performance. There is, of course, a complex 
relationship with the extent of federal or state interpenetration of 
the local school, but as Table 2 demonstrates, it is fair to character- 
ize most evaluation units as having minimal internal roles. Only in 
the area of in-service tra^ining do over h»lf of the evaluation units 
report having an internal organizational role. 

Insert Table 2 here 

Evaluation uniUp devote most of their time to generating information 

, t 

for groups internal to the organization (about 29% of their time), as com- 

N > ■ * ■ • ' . 

pared to less than 10% of their time devoted to generating information 

for external groups (federal or state agencjes; parents). However, as 

predicted ^ see -Proposition 6), actual use of the information generated 
^ <. ft - , _ 

is roughly equal externally and internally, depending on the particular 

measure. ... , 

As external use of the evaluation information- increases in importance 
relative to internal use, tbe unit. becomes 'increasingly loosely coupled, 
to the. rest of the organization, with organizational characteristics be- 
coming less important in predicting 'how effective the evaluation unit 
perceives itself to be in affecting use -of its results. Following Prop- 
position 7, the use of evaluation research becomes more problematic, both 
externally and internally. Incohsls.tent, ambiguous relations between 
different measures of use become apparent, and normal measures of per- 
formance of the evaluation unit, such as the range^of activities it per- 



fonns, become decoupled from the use of the information the evaluation 
unit generates. 

Insert Table 3 here 

Table 3 presents the correlation matrix for all variables used* as 
well as means and standard deviations. The first main results are Pre- 
sented in Table 4. This table is strongly supportive of the third hy- 
pothesis stated above*, the higher the proportion of local funds, the 
less the time spent on evaluation for external sources and the less the 
external use of evaluation results. Time spent on internal evaluation 
and iifvternal use is also nega /ely related, bu£ not as stv^ikingly. 
The evaluation unit staff is " ificantly smaller when local funds 
are a high propbrtion, though ovL.dll budget is not strongly related 
'(evaluation unit or district). Finally, as might be expected i -tH strict 
assessed value is positively related to extent of local support. 

. ............. "■ ■ . ^ 

Insert Table 4 here 

, , ... ... ^ 

• Table 5 presents some strong results on the internal role of the 
evaluation 4init. As Table 2 above shows, most evaluation units play mini 
mal roles in formulating internal school policy, even on issues directly 
related to the evaluation function. What is striking in Table 5 is the 
strong relationship between a performance measure, EU Act Range^ and 
internal role. While not as strong, the pattern of external or internjil 
time arid^use is equally striking. 'Internal role is positively related 
to external use time, but jiegatively related 'to internal use time, 
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as the second hypothesis above predicted. As might be expected, tight 
coupling in the district is directly related to an internal role for the 
evaluation unit. Finally, organizational characteristics are not strongly 
related to the presence of an internal role. 

V 

Insert Table 5 here 

Table 6 presents less consistent results. Rather than attempt an 
ad, hoc Interpretation at this point, the strong relationship between the 
amount of evaluation unit time spent for external purposes and. the ex- 
ternal use of evaluation results provides some independent validation of 
the variables created, 'However, the'^s^rong negative relationsip between 
the internal evaluation unit evaluation of federal programs <not outside 
counsultants br other school units) and external use is problematic. Fur- 
ther investigation of the external' use, measure seems appropriate, even 
though, this table presents results which are largely supportive of the 
argument made in this paper. " 

Insert.Table 6 here # 



NOTES 



[1] I am indebted to Marshall Meyer for suggestinig this inter- 
pretation. 

[2] Its -responsibility can be defined as "environment manage- 
ment," with few internal responsibilities (Zucker, 1979). 

0 £3] Spence assumes that 'education is a signal, that is, an 
alterable observable characteristic, while race is an index, that is, 
an unalterable observable characteristic. However, sociologists Wbuld 
argue that education will in most instances serve as an index (though 
affected by individual choice, the^ eiYiployer in fact assumes ft is 
ftxed at the current level), while'race is not uniformly an index, 
as the phenomenon of "passing" clearly illustrates. However, the 
willingness to identify and rely on signals or indices is of interest 
here* 
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Tible 1: Extent of st^tfi regul^tioo 4ad funding of local public 
schools and preaeACe of evaluation units CNsl32l)* 

Regulation/Funding if Districts Percent N 



High State 

IsTolveaent . 2^6' ^ 45.0 \ ' 502 
tfoderate State 

Involvement 136 25.7 529 
' Low State 



Involvement ^7 ^ 16.2 290 

* -Universe of 750 district witdi earollments- of 10,000 or more; 
50% saa^le of^districta \rCtlx 5»000 to 9,999 students (573). 
Response rate of 100% for larger districts (n=750); «1% for ' 
smaller districts (n*A6A) . /Table total n of 1321 must reflect 
telephone follow up rwults (Lyon, 1978). 



Table 2: Internal Role of Evaluation Units (Na227) 



Activity 


Percent 


. N 


Staff la-aervice 
Training 


66.5 


151 


Selection of Curricixlum/ 
Prograa Materials 


42.3 


96 


Y 

Allocation of ' 
» Funds 


36.6 


83 


Facilities 
Planning 


33.9 


77 


Collective 
Bargaining 


14.5 


. 33 


Teacher Performance 
Review 


9:7- 


22 



r 



If 




33 







2 


3 


4 


1. 


Local /Extarnal 
RasourcM 


^ -.007 


-.087 


.079 


2. 


Inrola 


X 


.177 


• 162 


3. 


External Uaa 


X 


X 


^346 


4. 


Intarnal Ua« 


X 


X 


X 


S. 


EV Tlnat Ext 


X 


X 


X 


6. 


EV TImat Int 


X. 


' X 


■ ...x 


7, 


Taac Uaat BxC 


'4 


sr 


X ^ 


8. 


EV Eval Fad 


X 


X 


X - 


9. 


EV Act Range 


X 




X 


10. 


*'Tlght'* Coupling 


X 


X 


X ' 


11. 


Consult Budgac 




. X 


X 


12. 


EV Staff 


"x 


X" 


X 


13. 


EV Budget 








U. 


DUC ^Budget * 








13. 


Dlat Value 










Mean 
S.D. 


2.01 


5.29 


9.6S 






.88 


.70 




1 




221 


250 





TAaU 3l CORRXUTIOM MATRIX 




9 


6 


7 


8 


9 


10 


?-.236 


.266 


• 030 


.160 




— • IM^J 


.163 


-•202 




• 084 


.365 


.051 














.262 


-.292 . 


-.115 


-.110 

1 


.179 


— .HAD 


-•OSS 


.068 


-.065 


.134 


' .053 




X 


.867 


.011 


-.205 


.136 


.105 


X 


X 


.031 


.207 
• 


-a77 


-.106 


X ' 


X 


X 


.018 


.021 


" .038 


X 


X- 


X 


X 


-.138 


-.089 


X K 


X 


X 


X 


X 


-•066^ 


X 


X 


X 


X 


X 




X 


X 


X . 


X 


X 


X 


X 

• 


^ X 


X 


X 

1 


X 


X 

* 1 




23.23 


1.47 


2.03 


* 

6.72 




6.23 . 


6.67 


• 75 


.82 


2.07"%, 




256 


256 


200 


259 


263 


\245 



\ 



11 


12 


13 


14 




0 

AD 


-.045 


-.373 


-.335 


-.268 


.110 


.171 


-.014 


.068 


.066 


-.076 


.063 


.153' 


.061 


.097 


.138 


. .028 


.086 , 


.102 


-.125 


.054 


.090 


-.116 


.152 


.058 


-.032 


.085 


.P85 


.124 


-.178 


-.032 


.056 


-.210 


7.154 


-.146 


.135 


.017 


.030 


■ .054 


.070 


.023 


-.061 


-.ibi 


.099.' 


-.202 


-/116 


-.109 


-.011 


-.018 


.096 


.300 


.267 


.06(1 I 


.126 


.230 


.030 


.056 


.051 


.142 


-.089 


-.009 


X 


.111. 


' .109 


.224 


.017 


.139 






.848 


."•If W 


177 


— .UO7 




. X 


X 


- .389 


.131 


-.052 






K 


X 


.OSS ' 


-.021 








' X 


' X 


X 



, J) 



3.08 4.77 .206.18 79.33 282.28 1.41 
7.90 9.94 439.72 219.25 242.54 .60 
241 262 247, .236. 186 244 



Table 4j Regreaaloa of extarxuX aad internal enphasia/uae of evaliutlon 
reaulta and organisational characterlatlcs on the ^xtent of local/external 

Maourcea (R a .21) 



External Uae 
Intenul Uae 
EU Tine: Ext 
lU Tina: Int 
EU Staff 
EU Budget . 
Oiat. Budget 
Olat. Value 



Unatandardized . - 

Regreaaion (Standard 
Coefflcienta Wtsota) 



Standardized 
Regreaaion , 
Coefflcienta 



-1.17 
-.55 
"2.13 
•1.61 
-.81 
-.01 
-.01 
.01 



(3.63) 
(4.62) 
(1.46) 
(1.42) 
(.55) 
(.01) 
(.05) 
(.01) 



.04 
.01 
.46 
.37 
.41 
.06 
.02 
.14 



F- ratios 



.10 
.01 
2.14 
1.30 
2.20 
.09 
.01 
1.58 



(Conatant 1.39) 



I 



Tabla 5} R«|raaslQa on sxtarnaX sad iatarnal emphati»/uaa of ovaliwtion 
raaults and organiajatlonal characteriatica on the externmX uao of 

avaluatlon reaulta (R a. 32) 





Unatandardlzed. 




Standardized 






Regreaaion 


(Standard 


Regreaaion 


F-catloa 


0 


Coeff identa 


£rrora) 


Coe££lclenta 


Ext«nial''nae 


.14 




,08 


.57 * 


Intanal Ua« 


-.13 


(.23) 


-.06 


.31- 


EU Tiaa: Ext 


.01 


(.07) 


.02 


- .01 


CU Tlae: Int 


-.05 


(.07) 


-.24 


.66 


iifiaX, Uae: Ext 


■> -.09 


(.22) 


-.04" 


.19 


Lbcal 


.13 


(,22) 


.06 


.34 


Etr Act Range 


.24 


(.08) 


.37 


9.48 


••tight" Couple 


.12 


(.17) 


-.07 


.53 


KChst#ff. 


-.01 


. (.02) 


-.12 


.33 


EU Budget 


.00 


(.00) 


.06 % 


.08 


(Conatant 1.82) 
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T«bl« 6i Ragrtisioa of txtcnial «acl lataroal •iiiph«ilft/u«« of •valuaittoa 
riiiultt «ad orgaalzational clur«ct«ristlca oa thm «xtira«l ui« of 

•▼«lumtloa raaulta (R » .26) 





unataadaedlzad 




Staadardlzfd 






Ragce»alon ^ 


(Standard 


Regr^salok 






Coaffloiaata 


Errors) 


Coefflcleats 


F-rat4iOa 


Intaznal Ua« 


.39 


(.15) 


,30 


7.10 


EO TIom: Ext 


.05 


(.05) 


.36 


1.39 


EOT^ffla: lAt 
T««t Use: Ext 


.01 


(.04) 


.07 


,04 


-c20 


(.14). 


-.15 


1.91 


EU Eval Fed 


-.18 


(.12) 


-.18 r 


2.29 


SO Act Range 


.01 


(.05) 


' .02 


.02 


£ocal/Ext Res., 


> .00 


(.00) 


.02 


:o3 . 


'*Tl^t" Coupla 


-.08 


(.11) 


-.08 


.55 


Conault Budget. 


.01 


(.01) 


.09 


.64 


EU Staff 


-.01 


(.02) 


-.10 


.16 


EU Budget 


.00 


(.00) 


.20 


,86 


(Cooatant 1.'37) 











